Smart photobook creation

ABSTRACT

A method for producing a photo album includes sorting images according to a primary predetermined criterion, separating the sorted images into a first page group and a second page group using one or more secondary criteria, and automatically selecting a first page layout from a library of page layouts. The first page layout includes a same number of one or more image receiving areas as the number of one or more images in the first page group. The one or more images in the first page group are automatically placed into the one or more image receiving areas in the first page layout.

The present application is a continuation application and claimspriority to commonly assigned pending U.S. patent application Ser. No.12/406,873, titled “Smart Photobook Creation”, filed Mar. 18, 2009, nowU.S. Pat. No. 8,131,114 the disclosure of which is incorporated hereinby reference.

TECHNICAL FIELD

This application relates to utilization of digital images, and morespecifically, to the design and creation of products based on digitalimages.

BACKGROUND OF THE INVENTION

In recent years, photography has been rapidly transformed from chemicalbased technologies to digital imaging technologies. Images captured bydigital cameras can be stored in computers and viewed on displaydevices. Users can also produce image prints based on the digitalimages. Such image prints can be generated locally using output devicessuch an inkjet printer or a dye sublimation printer or remotely by aphoto printing service provider. Other products that can be producedusing the digital images can include photo books, photo calendars, photomug, photo T-shirt, and so on. A photo book can include a cover page anda plurality of image pages each containing one or more images. Designinga photo album can include many iterative steps such as selectingsuitable images, selecting layout, selecting images for each page, addtext, and rearrange the images and text, which can be quite timeconsuming. It is desirable to provide methods to allow users to designand produce photo albums in a time efficient manner.

SUMMARY OF THE INVENTION

In one general aspect, the present invention relates to a method forproducing a photo album having a first page and a second page. Themethod includes sorting a plurality of images according to a primarypredetermined criterion to produce a list of sorted images; separatingthe sorted images into a first page group and a second page group usingone or more secondary criteria, wherein the first page group of one ormore images is to be placed in the first page of the photo album,wherein the second page group of one or more images is to be placed inthe second page of the photo album; graphically displaying the firstpage group of one or more images within a first border and the secondpage group of one or more images within a second border, wherein thefirst border and the second border respectively represent the first pageand the second page of the photo album; automatically selecting a firstpage layout from a library of page layouts, wherein the first pagelayout comprises a number of one or more image receiving areas thatmatch the number of one or more images in the first page group;automatically selecting a second page layout from the library of pagelayouts, wherein the second page layout comprises a number of one ormore image receiving areas that match the number of one or more imagesin the second page group; automatically placing the one or more imagesin the first page group into the one or more image receiving areas inthe first page layout; and automatically placing the one or more imagesin the second page group into the one or more image receiving areas inthe second page layout to produce the second page in the photo album.

In another general aspect, the present invention relates to a method forproducing a photo album having a first page and a second page. Themethod includes sorting a plurality of images according to a primarypredetermined criterion to produce a list of sorted images; separatingthe sorted images into a first page group and a second page group usingone or more secondary criteria, wherein the first page group of one ormore images is to be placed in the first page of the photo album,wherein each of the one or more images in the first page group has aportrait format or a landscape format, wherein the second page group ofone or more images is to be placed in the second page of the photoalbum; automatically selecting a first page layout from a library ofpage layouts, wherein the first page layout comprises a number of one ormore image receiving areas that match the number of one or more imagesin the first page group; automatically selecting a second page layoutfrom the library of page layouts, wherein the second page layoutcomprises a same number of one or more image receiving areas as thenumber of one or more images in the second page group, wherein the oneor more image receiving areas in the first page layout match theportrait format or the landscape format of the one or more images in thefirst page group; automatically placing the one or more images in thefirst page group into the one or more image receiving areas in the firstpage layout; and automatically placing the one or more images in thesecond page group into the one or more image receiving areas in thesecond page layout to produce the second page in the photo album.

In another general aspect, the present invention relates to a method forproducing a photo album. The method includes providing a plurality ofimages comprising a first image; associating text information with thefirst image; providing a library of page layouts each comprising one ormore image receiving areas, wherein at least one of the page layoutsincludes an image caption area associated with one of the imagereceiving areas in the page layout; after the step of associating,automatically selecting a first page layout from the library of pagelayouts, wherein the first page layout comprises an image receiving areaand an image caption area in association with the image receiving area;automatically placing the first image in the image receiving area in thefirst page layout; and automatically placing the text information intothe image caption area in association with the image receiving area inthe first page layout to produce a page in the photo album.

In another general aspect, the present invention relates to a method forproducing a photo album. The method includes providing a plurality ofimages including a first image; associating text information with thefirst image; selecting a first group of one or more images to be placedin a page of the photo album from the plurality of images, wherein thefirst group includes the first image; providing a library of pagelayouts each comprising one or more image receiving areas, wherein atleast one of the page layouts includes an image caption area associatedwith one of the image receiving areas in the page layout; after the stepof associating, automatically selecting a first page layout from thelibrary of page layouts, wherein the first page layout comprises animage receiving area and an image caption area in association with theimage receiving area, wherein the step of automatically selecting afirst page layout comprises matching the number of image receiving areasin the first page layout with the number of images in the first group;automatically placing the first image in the image receiving area in thefirst page layout; and automatically placing the text information intothe image caption area in association with the image receiving area inthe first page layout to produce the page in the photo album.

In another general aspect, the present invention relates to a method forproducing a photo album. The method includes providing a plurality ofimages including a first image; associating text information with thefirst image; selecting, from the plurality of images, a first group ofone or more images to be placed in a page of the photo album, whereinthe first group includes the first image, wherein the first group of oneor more images each has a format selected from the group consisting ofportrait, landscape, round, oval, and diamond; providing a library ofpage layouts each comprising one or more image receiving areas, whereinat least one of the page layouts includes an image caption areaassociated with one of the image receiving areas in the page layout;after the step of associating, automatically selecting a first pagelayout from the library of page layouts, wherein the first page layoutcomprises an image receiving area and an image caption area inassociation with the image receiving area, wherein the one or more magereceiving areas in the first page layout are each associated with aformat selected from the group consisting of portrait, landscape, round,oval, and diamond, wherein the first page layout is automaticallyselected in part by matching the number of image receiving areas in thefirst page layout with the number of images in the first group and bymatching formats of the one or more image receiving areas in the firstpage layout with formats of the one or more images in the first group;automatically placing the first image in the image receiving area in thefirst page layout, wherein the image receiving area has the same formatas the format of the first image; and automatically placing the textinformation into the image caption area in association with the imagereceiving area in the first page layout to produce the page in the photoalbum.

In another general aspect, the present invention relates to a method forproducing a photo album. The method includes providing a library of pagelayouts each comprising one or more image receiving areas eachassociated with a format selected from the group consisting of portrait,landscape, round, oval, and diamond; selecting, from a plurality ofimages, a first group of one or more images to be placed in the firstpage of the photo album, wherein the first group of one or more imagescomprises a first image, wherein the first group of one or more imageseach has a format selected from the group consisting of portrait,landscape, round, oval, and diamond; selecting, from the plurality ofimages, a second group of one or more images to be placed in the secondpage of the photo album; graphically displaying the first group of oneor more images within a first border that represents a first page;graphically displaying the second group of one or more images within asecond border that represents a second page; automatically selecting afirst page layout from the library of page layouts, wherein the firstpage layout comprises one or more image receiving areas, wherein thenumber of image receiving areas in the first page layout is matched tobe the same as the number of images in the first group; andautomatically placing the first group of one or more images into the oneor more image receiving areas in the first page layout to produce thefirst page in the photo album.

In another general aspect, the present invention relates to a method forproducing a photo album. The method includes providing a library of pagelayouts each comprising one or more image receiving areas eachassociated with a format selected from the group consisting of portrait,landscape, round, oval, and diamond; selecting, from a plurality ofimages, a first group of one or more images to be placed in the firstpage of the photo album, wherein the first group of one or more imagescomprises a first image, wherein the first group of one or more imageseach has a format selected from the group consisting of portrait,landscape, round, oval, and diamond; selecting, from the plurality ofimages, a second group of one or more images to be placed in the secondpage of the photo album; graphically displaying the first group of oneor more images within a first border that represents a first page;graphically displaying the second group of one or more images within asecond border that represents a second page; automatically selecting afirst page layout from the library of page layouts, wherein the firstpage layout comprises one or more image receiving areas, wherein theformats of the one or more image receiving areas in the first pagelayout are matched with the formats of the one or more images in thefirst group; and automatically placing the first group of one or moreimages into the one or more image receiving areas in the first pagelayout to produce the first page in the photo album.

In another general aspect, the present invention relates to a method forproducing a photo album. The method includes providing a library of pagelayouts each comprising one or more image receiving areas eachassociated with a format selected from the group consisting of portrait,landscape, round, oval, and diamond; selecting, from a plurality ofimages, a first group of one or more images to be placed in the firstpage of the photo album, wherein the first group of one or more imagescomprises a first image, wherein the first group of one or more imageseach has a format selected from the group consisting of portrait,landscape, round, oval, and diamond; selecting, from the plurality ofimages, a second group of one or more images to be placed in the secondpage of the photo album; graphically displaying the first group of oneor more images within a first border that represents a first page;graphically displaying the second group of one or more images within asecond border that represents a second page; automatically selecting afirst page layout from the library of page layouts, wherein the firstpage layout comprises one or more image receiving areas, wherein theformats of the one or more image receiving areas in the first pagelayout are matched with the formats of the one or more images in thefirst group; automatically placing the first group of one or more imagesinto the one or more image receiving areas in the first page layout toproduce the first page in the photo album; automatically selecting asecond page layout from the library of page layouts, wherein the secondpage layout comprises one or more image receiving areas, wherein theformats of the one or more image receiving areas in the second pagelayout are matched with the formats of the one or more images in thesecond group; and automatically placing the second group of one or moreimages into the one or more image receiving areas in the second pagelayout to produce the second page in the photo album.

In another general aspect, the present application relates to a methodfor producing a photo album having a plurality of pages. The methodincludes selecting a plurality of images each characterized by an imageformat selected from the group of a portrait format and a landscapeformat; providing a library of page layouts each comprising one or moreimage areas configured to receive the selected images, wherein each ofthe image areas is characterized by an image area format consistent withimage format; automatically selecting one or more page layouts from thelibrary of page layouts in accordance with the selected images and theirassociated image formats; and disposing the selected images in the imageareas on the one or more page layouts to produce one or more pages inthe photo album.

In another general aspect, the present application relates to a methodfor producing a photo album having a plurality of pages. The methodincludes selecting a plurality of images each characterized by an imageformat selected from the group of a portrait format and a landscapeformat; providing a library of page layouts each comprising one or moreimage areas configured to receive the selected images, wherein each ofthe image areas is characterized by an image area format consistent withimage format; automatically selecting one or more page layouts from thelibrary of page layouts in response to the selected images and theirassociated image formats; automatically matching the image formats ofthe selected images with the image area formats of the image areas inthe page layouts; and disposing the selected images in the image areason the one or more page layouts to produce one or more pages in thephoto album.

In another general aspect, the present application relates to a methodfor producing a photo album having a plurality of pages. The methodincludes selecting a plurality of images each characterized by an imageformat selected from the group of a portrait format and a landscapeformat; specifying a sequence for the selected images; providing alibrary of page layouts each comprising one or more image areasconfigured to receive the selected images, wherein each of the imageareas is characterized by an image area format consistent with imageformat; automatically selecting one or more page layouts from thelibrary of page layouts in response to the sequence and the imageformats of the selected images; and disposing the selected images in theimage areas on the one or more page layouts to produce one or more pagesin the photo album.

The disclosed methods and systems provide convenient ways for a user tocreate photo books. Images can be automatically sorted and groupedaccording to page breaks. Page layout selections can be automaticallygenerated to match the properties of the image on a page to save user'stime. Text information can be stored in association with the images, andcan be automatically arranged with its associated image in the pagelayout. The primary predetermined criterion is based on capture times ofthe images, wherein the sorted images are sequenced chronicallyaccording to the capture times of the images.

Implementations of the system may include one or more of the following.The one or more secondary criteria can be based on color content in theimages. One or more of the sorted images can be placed in the first pagegroup if popular colors in the one or more of the sorted images arewithin a predetermined distance in a color space. The popular color ofan image can be defined by a color region in the color space, whereinthe image includes at least a predetermined percentage of pixels havingtheir associated pixel values in the color region. The one or moresecondary criteria can be based on capture times of the images. One ormore of the sorted images can be placed in the first page group if theone or more of the sorted images are within a predetermined interval ofimage capture time. The one or more secondary criteria can be based onimage-capture locations associated with the images. One or more of thesorted images can be placed in the first page group if the one or moreof the sorted images are within a predetermined proximity inimage-capture location. The library of page layouts each can include oneor more image receiving areas each associated with a portrait format ora landscape format, wherein the one or more image receiving areas in thefirst page layout have matching formats as the one or more images in thefirst page group. The method can further include moving an image withinthe first border into the second border to move the associated imagefrom the first page to the second page. The first page layout caninclude an image caption area associated with one of the image receivingareas in the first page layout. The method can further includeassociating text information with an image in the first page group;automatically placing the image in the one of the image receiving areasin the first page layout; and automatically placing the text informationinto the image caption area in association with the image receiving areain the first page layout.

Implementations of the system may include one or more of the following.The text information associated with the first image can be provided bya user. The method can further include storing the first image and thetext information in an integrated data structure in a computer memory.The method can further include automatically displaying the first pagehaving the first image placed in the image receiving area and the textinformation placed in the image caption area in the first page layout.The method can further include selecting a first group of one or moreimages to be placed in the page of the photo album from the plurality ofimages, wherein the first group includes the first image. The method canfurther include sequencing the plurality of images to produce an imagelist, wherein the first group of one or more images is selected from thebeginning of the image list. The method can further include removing thefirst group of one or more images from the image list after the step ofautomatically placing the first image. The step of automaticallyselecting a first page layout can include matching the number of imagereceiving areas in the first page layout with the number of images inthe first group. The first group of one or more images can each have aformat selected from the group consisting of portrait, landscape, round,oval, and diamond. The one or more mage receiving areas in the firstpage layout can be each associated with a format selected from the groupconsisting of portrait, landscape, round, oval, and diamond, wherein thestep of automatically selecting a first page layout comprises matchingformats of the one or more image receiving areas in the first pagelayout with formats of the one or more images in the first group. Thestep of automatically placing the first image can include selecting animage receiving area in the first page layout that has the same formatas the format of the first image.

Implementations of the system may include one or more of the following.The method can further include moving an image within the first borderinto the second border to move the associated image from the first pageto the second page. The step of automatically selecting a first pagelayout can include matching the formats of the one or more imagereceiving areas in the first page layout with the formats of the one ormore images in the first group. The method can further includeautomatically displaying the page having the first group of imagesplaced in the one or more image receiving areas if the first pagelayout. The method can further include sequencing the plurality ofimages to produce an image list, wherein the first group of one or moreimages is selected from the beginning of the image list. The method canfurther include removing the first group of one or more images from theimage list after the step of automatically placing the first group ofone or more images. The first page layout can include an image captionarea associated with one of the image receiving areas in the pagelayout. The method can further include associating text information withthe first image; and after the step of automatically selecting a firstpage layout, automatically placing the text information into the imagecaption area in association with the image receiving area in the firstpage layout. The method can further include storing the first image andthe text information in an integrated data structure in a computermemory.

DRAWING DESCRIPTIONS

The accompanying drawing, which are incorporated in and form a part ofthis specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention:

FIG. 1 is a block diagram of a computer system.

FIGS. 2A and 2B respectively show a perspective view and an inside viewof a photo album.

FIG. 3A shows a flow chart for creating a photo album in accordance toone aspect of the application.

FIG. 3B shows a flow chart for creating a photo album in accordance toanother aspect of the application.

FIG. 4 illustrates a collection of images to be selected.

FIG. 5 illustrates an image list selected from the collection of imagesin FIG. 4.

FIGS. 6( a)-6(i) illustrate a library of page layouts having differentnumber of images or different image formats.

FIG. 7A illustrates an exemplified user interface for selecting thenumber of images on a page of a photo album.

FIG. 7B illustrates another exemplified user interface for selecting thenumber of images on a page of a photo album.

FIG. 8 shows a flow chart for creating a photo album in accordance toanother aspect of the application.

FIG. 9 shows a flow chart for creating a photo album in accordance toanother aspect of the application.

FIG. 10 shows a flow chart for creating a photo album in accordance toanother aspect of the application.

FIG. 11 shows a flow chart for creating a photo album in accordance toanother aspect of the application.

FIGS. 12A and 12B illustrate pages including image caption areas.

FIGS. 13A and 13B illustrate pages having different backgrounds.

FIG. 14 shows a flow chart for creating a photo album in accordance toanother aspect of the application.

FIG. 15 illustrates sorting and grouping of images by image capture timein accordance to an aspect of the application.

FIG. 16 illustrates sorting and grouping of images by image colorcontent in accordance to another aspect of the application.

FIG. 17 illustrates sorting and grouping of images by image-capturelocation in accordance to another aspect of the application.

DETAILED DESCRIPTION OF THE INVENTION

In the present invention, the term “photo album” refers to a book thatincludes one or two cover pages and one or more image pages. Each of theimage pages can include one or more images. The image pages can alsoinclude text or image caption. The image layout can also include abackground pattern. In the imaging industry, photo albums can includephoto books, scrapbooks, snap books, and any books form including boundimage pages.

A computer system 100 illustrated in FIG. 1 represents an exemplifiedhardware setup for executing software that allows a user to performtasks such as communicating with other computer users, accessing variouscomputer resources, and viewing, creating, or otherwise manipulatingelectronic content, that is, any combination of text, images, movies,music or other sounds, animations, 3D virtual worlds, and links to otherobjects. The system includes various input/output (I/O) devices (mouse103, keyboard 105, display 107) and a general purpose computer 100having a central processor unit (CPU) 121, an I/O unit 117 and a memory109 that stores data and various programs such as an operating system111, and one or more application programs 113. The computer system 100also typically includes non-volatile memory 110 (e.g., flash RAM, a harddisk drive, and/or a floppy disk or other removable storage media) and acommunications card or device 123 (e.g., a modem or network adapter) forexchanging data with a network 127 via a communications link 125 such asa cable modem, DSL service or wireless Internet connection.

The computer 100 of FIG. 1 also can be connected to various peripheralI/O devices. One of the more popular of such peripheral devices is adigital camera 108. The digital camera 108 enables users to takepictures (i.e., images), which are saved in memory (not shown) withinthe digital camera 108 in a digital (electronic) format. After takingand storing the images, the user can connect the digital camera 108 to acomputer system 100 in order to upload the digital images to thecomputer's disk drive or other non-volatile memory 110. The user canalso wirelessly transfer digital images in the digital camera to thecomputer system 100. Once the digital images are uploaded to thecomputer system 100, the user can erase the digital images from thememory of the digital camera 108 so that the user can take and storeadditional images using the digital camera 108. Typically, the digitalcamera 108 is connected to the computer 100 only while the user isuploading images to the computer's disk drive or other non-volatilememory 110.

Users also can obtain digital images, for example, of film-based printsfrom a traditional camera, by sending an exposed film into aphoto-finishing service, which develops the film to make prints and thenscans (or otherwise digitizes) the prints or negatives to generatedigital image files. The digital image files then can be transmittedback to the user by e-mail or on a CD-ROM, diskette, or other removablestorage medium.

Once the digital images are stored on the computer 100, a user canperform various operations on them. For example, an image viewerapplication can be used to view the images. A photo editing applicationcan also be used to touch-up or otherwise modify the images. Inaddition, an electronic messaging (e.g., e-mail) application can be usedto transmit the digital images to other users.

In addition to viewing the digital images on the computer display 107,users often desire to have physical products based on the digitalimages. Such hard copies can be generated locally by the user usingoutput devices such an inkjet printer or a dye sublimation printer 130or remotely for a photo printing service provider. Other products basedon the digital images can include photo books, photo calendars, photomug, photo T-shirt, and so on. A photo book can include a cover page anda plurality of image pages each containing one or more images. Each pagecan also include different formats. Designing a photo album can includemany iterative steps such as selecting suitable images, selectinglayout, selecting images for each page, add text, which can be quitetime consuming. It is desirable to provide methods for designing andproducing photo albums that can substantially save a user's time. FIGS.2A and 2B shows an exemplified photo album 200 comprising a plurality ofmany pages 210, each including one or more images 220 provided by auser. The user can select a layout for each page and select images 220for each page.

Referring to FIG. 3A, a photo album can be created in the followingsteps: images can be captured by taking pictures using a digital camera,copying existing images through the Internet, or by any other methods.The captured images are then transferred to a computer and stored in amemory on the computer (step 310). A user selects a list of images onthe computer system 100 to be used in a photo album (step 320).Referring to FIG. 4, a collection 400 of images is shown in a graphicuser interface as candidates to be incorporated into a photo album. Byclicking on the small square below each of the images, the images can beselected into an image list 500 (shown in FIG. 5). More images can beviewed and selected into the image list by clicking a “next page” button410 or a “previous page” button 420.

Referring to FIG. 5, an image list 500 can therefore created by theuser. The user interface in FIG. 5 shows an array of thumbnail imagesthat represent the original images (i.e. full size images) provided bythe user. The image list 500 can reside on different view fields whichcan be viewed by clicking the “next page” button 510 or the “previouspage” button 520. In some embodiments, the image list 500 can besequenced in a same order as how they will be incorporated into a photoalbum. The sequence of images in the image list 500 can chronicallyordered by capture time. Alternatively, the images can also be groupedand ordered by events and activities, which may be optimal for telling astory using the photo album.

Referring back to FIG. 3A, a user can select the images to be displayedon a page of the photo album in a number of different ways. For example,referring to FIG. 7A, a user can select, by clicking a button on a userinterface, how many images are to be displayed on the next page of aphoto album. For example, there can be one, two, and three images on apage. The number of images per page can vary in a photo album. In somecases, a user may select a constant number of images for each page of aphoto album.

In another example, the image list 500 (FIG. 5) can be separated by pagegroups. As shown in FIG. 7B, images 731-735 that represent the originalimages in the image list 500 can be graphically grouped in two pagegroups: Page 10 and Page 11 in a graphic user interface 700. The images731, 732 in Page 10 can be surrounded by a border 740 to clearlyseparate them from the images 733-735 that are positioned within aborder 741. Each page can include different numbers of images, anddifferent types (portrait, landscape, round, oval, diamond, square,etc.) images. A landscape format describes images that have longerlateral dimensions than their vertical dimensions. A portrait formatrefers to images that have longer vertical dimensions than lateraldimensions. A user can flexibly move images 731-735 between differentpage groups. Buttons 710 and 720 can help a user to browse all the pagesin the photo album.

A user can readily re-arrange images between different pages using theuser interface shown in FIG. 7B. For example, a user can move image 732from page 10 (within border 740) to page 11 (within border 741). Theuser can also move image 733 from page 11 (within border 741) to page 10(within border 740). The user can easily add or remove images from theborders 740 or 741 and the pages they respectively represent by movingimages from or to the image list. The number of images within eachborder and their associated pages can thus be easily adjusted by theuser. The page layouts, as described below, can be automaticallyselected in response to the dynamic changes in the image compositions inthe borders 740 and 741.

Referring back to FIG. 3A, a library of page layouts are provided forcreating pages in a photo album (step 315). A page layout defines one ormore image areas on a page in the photo album. An image area can have arectangle shapes such as landscape or portrait format, or square. Animage area can also have round, shape, diamond, or other shapes. A pagelayout can have different numbers of images which may have differentcombinations of landscape and portrait formats for the image areas.

FIGS. 6( a) to 6(i) illustrate exemplified page layouts for a photoalbum. The page layout shown in FIG. 6( a) contains a single image area611 for receiving an image in landscape format. FIG. 6( b) shows a paylayout containing an image area 612 suitable for a portrait image. Apage layout in FIG. 6( c) includes portrait image areas 621 and 622,while the page layout in FIG. 6( d) contains landscape areas 623 and624. FIG. 6( e) shows a layout that includes a portrait image area 625and a landscape area 626. FIG. 6( f) shows a pay layout having threelandscape areas 631, 632, and 633, while FIG. 6( g) shows three portraitareas 634, 635 and 636 in a page layout. FIG. 6( h) shows two landscapeareas 637, 638 and a portrait area 639 in a page layout. FIG. 6( i)shows two portrait areas 641, 642 and a landscape area 643 in a pagelayout.

In accordance with the present invention, the disclosed system andmethods can automatically select a page layout from the library of pagelayouts in accordance to the properties of the images to be displayed ona page (step 330). For each page in the photo album, the page layoutselected from the library of page layouts should include the same numberof images to be displayed on the next page. The page layout selectedshould also include the same numbers of portrait and landscape imageareas as the numbers of portrait and landscape images to be displayed onthe next page.

Since it is desirable to provide a large number of page layouts,multiple page layouts in the page layout library may satisfy the numberand format of the groups of images to be displayed on the next page. Forexample, five page layouts may exist that all include two imagereceiving areas with one portrait and one landscape format, as requiredby the group in Page 10 (shown in FIG. 7B). An algorithm stored in thecomputer or at a remote server can automatically select one of themultiple choices from the satisfactory page layouts. For example, themost popular page layout among five satisfied page layouts can beselected. The user can change the page layout later when the user finetunes the designs of the photo album.

Once an appropriate page layout is selected for the images to bedisplayed on the next page, referring back to FIG. 3A, the selectedimages can be automatically placed onto the selected page layouts tocreate pages (step 335). Image cropping may be needed in some situationswhen the aspect ratio or shape of an image receiving area in a pagelayout does not exact fit a selected image. The image cropping can beautomatically conducted by a computer or determined by a user.

An electronic file containing the designs for the photo album can beproduced and stored on the computer 100 (step 340). A physical (i.e.hardcopy) photo album (or photobook) can be manufactured by a digitalprinter and other finishing equipment in accordance to the designsdefined by the electronic file.

Other detailed examples about the selections of images for a page andthe subsequent automatic selection of page layout (steps 325 and 330)are illustrated in FIGS. 8-11. In FIG. 8, a single picture is selectedfor a page. The image list and the library of page layouts are provided(steps 801 and 802). The disclosed system can automatically select fromthe top of the image list 500 (step 803). The selected image is removedfrom the image list 500 (step 804). The format of the selected image isnext checked (step 805) if it is of a landscape or a portrait format, orother shapes such as round, oval, diamond, etc. If the selected imagehas a landscape format, the image is dropped in the image area 611 ofthe page in FIG. 6( a) in step 806. If the selected image is portrait,the disclosed system drops the image in the image area 612 of the pagein FIG. 6(2) in step 807. In step 808, the disclosed system adds thepage that has just been processed in either step 806 or step 807 to thephoto album 200 that is being created. In step 809, the disclosed systemchecks if the image list 500 is empty. If the image list 500 is empty,it means all images in the list have been added to the photo album 200.The disclosed system then saves the complete photo album 200 on thecomputer. The photo album can be produced by a digital photo printer andother equipment (step 810). If the image list 500 is not empty, thereare unused images on the list to be added to the photo album 200. Inthis case, the disclosed system goes back to step 803 to select the nextavailable image at the top of the image list 500. This loop is continueduntil all images in the image list 500 are used and added to the photoalbum 200.

In FIG. 9, two images are selected to be displayed on a page of thephoto album. The disclosed system selects up two images from the imagelist 500 to dispose in the image areas (step 903). The selected imagesare removed from the image list 500. In step 905, the disclosed systemchecks if the two selected images are both landscape images. If theyboth are, the disclosed system randomly drops the two images into imageareas 623 and 624 in FIG. 6( d). If the check in step 905 is “No”, thedisclosed system checks in step 906 if both selected images are portraitimages. If they both are portrait images, the disclosed system randomlydrops the two images into the image areas of 621 and 622 in FIG. 6( c).If the check in step 906 is “No”, the only possibility is, one of theselected images is landscape image and the other is portrait image. Inthis case, the disclosed system drops the portrait image in the imagearea 625 and the landscape image in the image area 626 in FIG. 6( e) instep 909. When either steps 907, 908 or 909 is finished, the disclosedsystem, in step 910, adds the page created in either steps 907, 908 or909 to the photo album 200 that is being created. Then in step 911, thedisclosed system checks if the image list 500 is empty. If the imagelist 500 is empty, it means all images in the list have been added tothe photo album 200. The disclosed system then sends the complete photoalbum 200 to a digital photo printer to print out the photo album instep 912. If the image list 500 is not empty, there are unused images onthe list to be added to the photo album 200. In this case, the disclosedsystem goes back to step 903 to select the next two available images atthe top of the image list 500. This loop is continued until all imagesin the image list 500 are used and added to the photo album 200.

When the computer system picks up the next two available images from theimage list 500. Toward the end of the image list 500, there can beeither one image left in the image list 500, or there can be two. If twoimages remain in the image list 500, the disclosed system picks up thelast two images finishes the photo album (step 912). If the image list500 only includes a single remaining image, the disclosed system picksup the last image and treats the second empty image as having the sameformat as the last available image. The final finished photo album 200can have an empty image area on the last page. The disclosed system canalso automatically adjust the last page to include only one imagereceiving area.

In FIG. 10, three images are selected to be displayed on a page of thephoto album. An image list 500 and a page layout library are first takenby the disclosed system (not shown in FIG. 10). In step 1003, thedisclosed system selects up three images from the image list 500. Theselected images are then removed from the image list 500. In step 1005the disclosed system checks if all three selected images are oflandscape format. If the check 1005 is “Yes”, the disclosed systemrandomly drops the three selected images into 631, 632 and 633 of FIG.6( f). If the check 1005 results in “No”, the disclosed system checks,in step 1006, if all three selected images are of portrait format. Ifthe check 1006 results in “Yes”, the disclosed system, in step 1009,randomly drops the three selected images into 634, 635 and 636 in FIG.6( g). If the check 1006 is “No”, the disclosed system, in step 1007,checks if two of the three selected images are of landscape format andthe third selected image is of portrait format. If the check 1007results in “Yes”, the disclosed system, in step 1010, drops randomly thetwo landscape images into 637 and 638 and the one portrait image in 639in FIG. 6( h). If the check 1007 results in “No”, the only possiblecombination is that two of the selected images are portrait images andthe third one landscape. The disclosed system, in step 1011, randomlydrops the two portrait images into 640, and 641, and the landscape imagein 642 of FIG. 6(I). With either of the steps 1008-1011 is finished, thedisclosed system adds the current page to the photo album 200 in step1014 followed by a check 1012 to see if the image list 500 is empty. Ifthe check 1012 is “Yes”, all images in the image list 500 have beenadded to the photo album 200, and the disclosed system sends the photoalbum to a digital printer in step 1013. If the check 1012 results in“No”, there are unused images in the image list 500, and the disclosedsystem goes back to step 1003. This loop is on until all images in theimage list 500 are used in the photo album 200.

As the image list may emptied out as pages in the photo album is filled.There may be 1, 2 or 3 remaining images in the image list 500. In thecase there are three remaining images in the image list 500, thedisclosed system goes through the ordinary process and selects the lastthree images. In the case that there is one remaining image in the imagelist 500, there are two possibilities. If the remaining image islandscape, the disclosed system produces the last page of the photoalbum 200 using the page layout of FIG. 6( f), with 631 filled with thelast image in the image list 500, and 632 and 633 empty. If theremaining image is portrait, the disclosed system produces the last pageof the photo album 200 using the page layout of FIG. 6( g), with 634filled with the last image in the image list 500, and 635 and 636 empty.In the case there are remaining images in the image list 500, there arethree different situations. (1) Both of the remaining images arelandscape images. (2) Both of the remaining images are portrait images.(3) One of the remaining images is landscape image and the otherportrait. For situation (1), the disclosed system uses the page layoutof FIG. 6( f), filling 631 and 632 with the two remaining images, andleaving 633 empty. For situation (2), the disclosed system uses the pagelayout of FIG. 6( g), filling 634 and 635 with the two remaining images,and leaving 636 empty. For situation (3), the disclosed system used thepage layout of FIG. 6( h), filling 637 with the remaining landscapeimage and 639 with the remaining portrait image and leaving 638 empty.For the cases where there are one or two empty image areas in the lastpage of the photo album 200, the user can either leave the finishedphoto album 200 as it is, or manually fill the open image area(s) withadditional image(s).

A user can also select different numbers of images for a page in thephoto album, such as a combination of one, two, or three images perpage, as shown in FIG. 11. In step 1103, the disclosed system randomlyselects a number among 1, 2 or 3 for the number of images on the currentpage of the photo album 200 that is being created. After the number ofimages is selected, the disclosed system selects that number of imagesfrom the image list 500. The selected images are then removed from theimage list 500 (step 1104). The disclosed system then checks if a singleimage has been selected (step 1105). If the result is “Yes”, theselected image is dropped in the current page (step 1107) (which caninclude steps 805, 806, and 807 as shown in FIG. 8). If the result is“No” in step 1105, the disclosed system further checks if two imageshave been selected (step 1106). If the check 1106 results in a “Yes”,the disclosed system drops the two selected images in the current pagein step 1108, which includes steps 905-909 in FIG. 9. If the check 1106results in a “No”, three images must have been picked. In thissituation, the disclosed system drops the three selected images to thecurrent page in step 1109, which includes steps 1006, 1007, 1009-1011 inFIG. 10. After either of steps 1107, 1108, or 1109 is finished, thedisclosed system adds the current page to the photo album 200 in step1110. Then the disclosed system checks in step 1111 if the image list500 is empty. If the image list 500 is empty, all images have been addedto the photo album 200. The disclosed system sends the photo album 200to a digital printer in step 1112 to print out a hard copy of the photoalbum. If the image list 500 is not empty, there are unused imagesremaining to be added to the photo album 200. The computer goes to step1103 and select more images to process. The loop can continue until allimages in the image list are incorporated into the photo album 200.

The page layouts can include text field for receiving text from user.The text can be image caption associated with each image on the page.The text can also be a general caption on the page. The text can includea title for the front or cover page of the photo album. As shown FIGS.12A and 12B, captions area 1201 and 1202 is provided in page layoutsnext to their respective image receiving areas 1203 and 1204. A pagelayout can also be featured with some pre-defined background images. Forexample, FIG. 13A has a background 1301 with a house, some cloud and asun in the ski, while FIG. 13B has a background 1302 with some balloons.The user can change the background for each and every page of the photoalbum 200. The background images can also be automatically selected bythe computer in response to the image content, the title of the photoalbum or the captions for the images entered by the user.

In some embodiments, referring to FIG. 3B, after a list of images isselected (step 360), text information can be entered by the user inassociation with an images in the image list (step 365). The textinformation can include description or remarks about an image. The textassociated with an image is stored as an integrated data structure in acomputer memory. When an image is selected from the image list (step370), the text information associated with the selected image isautomatically captured together with the selected image. A library ofpage layouts are separately proved (step 372). Next, a page layout isautomatically selected from the library of page layouts in accordancewith the properties of the images on the page as well as the textinformation associated with the image receiving areas in the page layout(step 375). The format of the image receiving field in the page layoutmatches the format of the selected image. The page layout also includesan image caption area positioned adjacent to the image receiving areafor receiving the associated text information. For example, one portraitimage and one landscape image may be selected for the next page. Theportrait image has associated text information entered by the user andstored in conjunction with the portrait image. The landscape is notassociated with any text information. In selecting a page layout, thedisclosed system first select page layouts containing two imagereceiving areas, and then further selects page layouts that have includeone portrait and one landscape image receiving areas. Additionally, thedisclosed system further selects page layouts that include an imagecaption area in association with the portrait image receiving area.After the page layout is selected, the selected image(s) and associatedtext are automatically placed in the selected page layout (step 380).The placement of the selected image(s) and associated text can be donein a single step because they are pre-stored in an integrated datastructure as described previously. The same steps can be repeated forfill the image fields and text fields in each page of a photo book tocreate the photo book (step 385).

In some embodiments, referring to FIG. 14, a plurality of images isstored on a computer system (step 1410). The images can be first sortedby a computer system based a primary criterion (step 1415) to produce alist of sorted images. The images are sorted by a predetermined primarycriterion based on the likely sequence that the images will be presentedin a photo book (step 1415). For example, the images can be sorted bytheir associated image capture times. The image capture times can beextracted from the header information in the image files. For example, aJPEG image may include an EXIF header that includes the capture time ofthe image. It is stored by the digital camera in the image file asinformation in association with the image data. Referring to FIG. 15, acumulative image count can be plotted against image capture time. In oneimplementation, the list of sorted images can be sequenced chronicallybased on capture time of the images. Users often prefer to have imagesto appear in book pages in a chronicle order.

The list of sorted images is then separated into page groups by acomputer system using one or more secondary criteria (step 1420). Forexample, images can be grouped according to the capture times. Imagescaptured in a short time interval are more likely suitable to be placedon the same page, facing pages, or adjacent pages in a photo book. Insome embodiments, images that are taken in a short interval (e.g. a day)can be assigned to a same group (e.g. Group A, FIG. 15). The images inGroup A, Group B, or Group C (FIG. 15) are captured in a short timeinterval. The images in each of these groups can be respectively groupedas candidates to appear on a same page, facing pages, or adjacent pages.

The sorted images may be first placed into a group including a largenumber of images that do not fit on a page. For example, ten images inGroup B (FIG. 15) are taken in a short time interval such a day. Theimages in Group B can be further separated into page groups using othersecondary criteria such as color content and image capture locations.For instance, the most popular color of each of the ten images can beplotted in a color space (such as (Y, u, v), (L, a, b) etc), as shown inFIG. 16. The color space is first separated into finite number ofregions. The most popular color is defined as the color region that hasthe most pixel values that fall into. The ten images can be furthersub-grouped according to their adjacencies in the color plane (u, v):images 1-3 may be categorized into a first group; images 4-6 may becategorized into a second group; images 7-10 may be categorized into athird group. The images 1-10 in Group A might have been taken within asame day. The photographer might have traveled to several differentsurroundings, such as a beach, a restaurant indoor, or a swimming poolat the hotel, each of which may have different color distributions.These different groups of images can be displayed on different pagegroups, as shown in FIG. 7B (step 1425). For example, in the userinterface, image 1-3 can be bounded by a border 740 or 741 and to beassigned to Page 10. Images 4-6 can be displayed within the border 740or 741 and to be assigned to Page 11. Images 7-10 can be placed withinthe border 740 or 741 and to be assigned to Page 12. It should beunderstood that the display of color content distributions of images arenot limited to a specific color coordinates or the two dimensional colorplane shown in FIG. 16. The color distributions can be plotted in threedimensional color spaces (Y, u, v), (L, a, b), or two dimensional colorplanes such as (Y, u), (Y, v), (a, b), (L, a) etc.

In some embodiments, the images can be sorted or grouped based ongeographic locations. For example, assuming Group C (FIG. 15) includesfive images that are taken, for example, in a same day. Geo locationinformation about each of the five images can be extracted from theheader information of the five images, and plotted as shown in FIG. 17.Images 1-3 are shown to have been taken in close proximity while images4-5 are taken at similar locations. Images 1-3 can be sub-groupedtogether and assigned to Page 4 of the photo book. Images 4-5 can besub-grouped together and assigned to Page 5 of the photo book. The twogroups of images can be displayed in different borders 740, 741 on auser interface, as shown in FIG. 7B, to indicate that the images belongto different pages of a photo book. In some embodiments, the Images 1-3and the Images 4-5 can be assigned to be on facing pages (e.g. such aspages 4 and 5) such that the images from the same events can be viewedon facing pages in one open-book view. The placements of two page groupsof images on facing pages can be based on one or more secondarycriteria, such as image capture time, image color content, and imagecapture location.

A library of page layouts are separately provided (step 1427). Next, apage layout is automatically selected from the library of page layoutsin accordance with the properties of the images on the page as well asthe text information associated with the image receiving areas in thepage layout (step 1430). The format of the image receiving field in thepage layout matches the format of the selected image. For example,images in Group A (FIG. 15) may include two landscape images (L) and twoportrait images (P). A notation for the image formats for Group A can be“LLPP”. The page layout selected from the page-layout library shouldhave four image receiving areas with matching image formats “LLPP”.

The page layout also includes an image caption area positioned adjacentto the image receiving area for receiving the associated textinformation. For example, one portrait image and one landscape image maybe selected for the next page. The portrait image has associated textinformation entered by the user and stored in conjunction with theportrait image. The landscape is not associated with any textinformation. In selecting a page layout, the disclosed system firstselect page layouts containing two image receiving areas, and thenfurther selects page layouts that have include one portrait and onelandscape image receiving areas. Additionally, the disclosed systemfurther selects page layouts that include an image caption area inassociation with the portrait image receiving area. After the pagelayout is selected, the selected image(s) and associated text areautomatically placed in the selected page layout (step 1435). Theplacement of the selected image(s) and associated text can be done in asingle step because they are pre-stored in an integrated data structureas described previously. The same steps can be repeated for fill theimage fields and text fields in each page of a photo book. The user canselect an image from the plurality of images to be used as part of thebook cover. Finally, the photo book is created (step 1440).

It should be understood that the library of page layout contains layoutswith different numbers of images other than one, two, and three. For agiven number of images on a page and for the given formats of theimages, the layout is not limited to the designs shown in thisinvention. There can be a large variation of layout designs for the samenumber of images with the same formats on each page. Moreover, thesorting and grouping of images can be implemented by other approaches orusing other metrics. The illustration of page grouping and page breaksfor the images can also vary within the scope of the present disclosure.

What is claimed is:
 1. A method for producing a photo album having aplurality of pages, comprising: sorting a plurality of images accordingto a primary criterion to produce a list of sorted images; separatingthe sorted images into a first page group and a second page group usingone or more secondary criteria, wherein the first page group of one ormore images is to be placed in a first page of the photo album, whereinthe second page group of one or more images is to be placed in a secondpage of the photo album; storing a library of page layouts in a computersystem; automatically selecting a first page layout from the library ofpage layouts by the computer system, wherein the first page layout isconfigured to receive the number of one or more images in the first pagegroup; automatically selecting a second page layout from the library ofpage layouts by the computer system, wherein the second page layout isconfigured to receive the number of one or more images in the secondpage group; automatically placing the one or more images in the firstpage group in the first page layout to form at least a portion of thefirst page of the photo album; and automatically placing the one or moreimages in the second page group in the second page layout to form atleast a portion of the second page of the photo album.
 2. The method ofclaim 1, wherein the primary criterion is based on capture times of theimages.
 3. The method of claim 2, wherein the sorted images arechronically sequenced according to the capture times of the images. 4.The method of claim 1, wherein the one or more secondary criteria arebased on color content in the images.
 5. The method of claim 4, whereinone or more of the sorted images are placed in the first page group ifpopular colors in the one or more of the sorted images are within apredetermined distance in a color space.
 6. The method of claim 5,wherein a popular color of an image is defined by a color region in thecolor space, wherein the image includes at least a predeterminedpercentage of pixels having their associated pixel values in the colorregion.
 7. The method of claim 1, wherein the one or more secondarycriteria are based on capture times of the images.
 8. The method ofclaim 7, wherein one or more of the sorted images are placed in thefirst page group if the one or more of the sorted images are within apredetermined interval of image capture time.
 9. The method of claim 1,wherein the one or more secondary criteria are based on image-capturelocations associated with the images.
 10. The method of claim 9, whereinone or more of the sorted images are placed in the first page group ifthe one or more of the sorted images are within a predeterminedproximity in image-capture location.
 11. The method of claim 1, furthercomprising: graphically displaying the first page group of one or moreimages within a first border and the second page group of one or moreimages within a second border, wherein the first border and the secondborder respectively represent the first page and the second page of thephoto album.
 12. The method of claim 11, further comprising: allowing animage within the first border to be moved into the second border to movethe associated image from the first page to the second page.
 13. Themethod of claim 1, further comprising: associating text information withan image in the first page group; automatically placing the image in thefirst page layout; and automatically placing the text information inassociation with the image in the first page layout.
 14. The method ofclaim 1, wherein the one or more images in the first page group areautomatically placed into one or more image receiving areas in the firstpage layout, and wherein one or more images in the second page group areautomatically placed into the one or more image receiving areas in thesecond page layout.
 15. The method of claim 14, wherein the library ofpage layouts each comprises one or more image receiving areas eachassociated with a portrait format or a landscape format, wherein the oneor more image receiving areas in the first page layout have matchingformats as the one or more images in the first page group.
 16. Themethod of claim 15, wherein each of the one or more images in the firstpage group has a portrait format or a landscape format, and wherein theone or more image receiving areas in the first page layout match theportrait format or the landscape format of the one or more images in thefirst page group.
 17. The method of claim 1, further comprising: placingthe first page group of one or more images and the second page group ofone or more images on facing pages of the photo album based on the oneor more secondary criteria.
 18. The method of claim 1, furthercomprising: storing the photo album in the computer system.
 19. Themethod of claim 1, further comprising: making a physical photobook basedon the photo album using a digital photo printer and/or other equipment.